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This paper presents one researcher's premise that the 
most important variable in determining classroom effectiveness is the 
congruence of the delivered curriculum with the desired outcomes or, 
that students be given the opportunity to learn what is expected of 
them. This theory presupposes that curriculum expectations be made 
clear to students, and that students be givnn sufficient time and 
resources to achieve those expectations. The role of administrators 
is therefore to make these expectations clear to teachers, to provide 
teachers with the means to carry them out, and to assure that 
teachers do carry the goals out. Teachers and administrators must 
share a common belief that certain outcomes are expected for students 
at a given grade level^N^n a given school. Evaluation devices must 
also be geared precisely to these specific curricular goals* Threi 
important related concepts are defined and discussed; allocated time, 
engaged time, and success rate. These concepts are tl^en brought 
together into a single concept of academic learning time, defined as 
enq-jged time irith materials or activities that produce a high success 
r«tte and are related to outcome measures. This concept of academic 
learning time is shown '».o be a good predictor of classroom 
effectiveness. The paper concludes with general observations on the 
utility of research in assessing teacher effecti' en«ss. (TE) 
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In deciding what I can share with you from my research back- 
ground that might help you in your job of making schools more 
effective, I have decided to present a few rather simple propositions 
that are worth considering seriously as you visit classrooms. I 
understand most of you are principals and I think that you probably 
recognize that the whole movemesit toward effective schools rests on 
principals' shoulders. Whether you lifee it or not, you have ended up 
on the spot and it means a new role for some principals, an instruc- 
tional leadership rb^ie. If you have not been visiting classrooms a 
lot, you are going to be pressured to do so. And when you get into 
the classrooms, you neo i certain kinds of spectacles to look at what 
goes on. I am going to try to provide you with some of those 
spectacles today— what to loo*; for and what to think about as jou 
visit classrooms. 
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I boiled down a lot of what I wanted to say into a single 
variable or single statement. It seems to me that the single most 
important variable in determining whether or not you have an eftec- 
t-ive classroom is whether or not the delivered curriculum of that 
classroom is linked logically or empiricany to the outcomes that are 
desi red. This is such a deceptively simple statement that it 
obviously needs elaboration. First of all it is concerned with what 
is sometimes called curriculum alignment, the congruence of the 
curriculum with the outcome, the overlap of curriculum with outcomes, 
or as it is most often called in the research community, opportunity 
to learn. 




A student must have the opportunity to learn what it is that is 
expected of him or her. For certain subjects, it's crucial. You do 
not ordinarily learn trigonometry at home. The family is an 
important educator, but famlli^es do not teach trigonometry. There 
d'^e lots of school curriculum areas that families don't help with* 
Some social studies can be learned around the table at home, but 
cnemistry Isn't usually learned at home. There are parts of the 
school curriculum that will only be learned If we expose our children 
to It in schools. Otherwise, they will never get exposed to some 



^ things we deem valuable. 

^ What is expected of the student must be made clear to the 

student* The opportunity to learn it must be provided. If we have 
to make clear to the students what it is we expect of them, then 
please notice that what is expected must be clear to the teacher-- 
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crystal clear. A teacher who does not know what Is expected— what 
outcomes are desired--has been abandoned by the administrative staff 
of the district. That is a harsh statement. But if I talk to a 
\ teacher who says, "I don't know what anybody wants around here", I'm 

pretty sure where the blame is. I think it has to do with the 
administrative staff of a district not taking responsibility for 
saying, "this is what's expected." In addition, I would add that a 
teacher who knows the outcomes that are desired, but doesn't work 
toward them, is probably unmonltored by the administrative team of a 
district. Monitoring whether classes are doing work related to ^ 
Outcomes that are valued is a proper concern of administrators. If 
teachers do not know or do not work toward desired outcomes, what 
takes place Is more like babysitting than It Is education. Some of 
the oldest issues in education revolve around this problem. Ue need 
to know what knowledge or skills are worth acquiring. To repeat the 
title of a famous article at the turn bf the century, we need to know 
"What knowledge is . of most worth." The teachers and the administra- 
tors_ of a school district simply have to share a common belief that 
certain outcomes are expected for students at a given grade level in 
a given school. 

I would remind you that the whole concept of effectiveness which 
brings you here today hinges on valuing some set of outcomes. You ; 
cannot talk about effectivenessunLess you talk about what it is you 
are trying to teach. The(oyt€t5ines issue Is primary in discussions of 
effectiveness. Good teacnTng is a logical issue. It need not pro- 
duce learning. A teacher who starts a lecture on time, provides a 
review, gives an advance organizer, emphasizes important points, asks 
higher order questions throughout, summarizes key issues, cracks a 
good joke, etc., may be judged to be a good teacher whether or not 
the students learn. Good and poor teaching is determined by values 
and knowledge of what standards of practice are. A doctor may have 
patients that die, but if the doctor used the best practices, she may 
be judged to be a good physician. An effective physician however, is 
associated with many fewer deaths. The outcomes for medicine are 
manifestly clear, and effectiveness is, therefore, easy to judge. 
The outcome for educators must also be made clear or we will not ever 
be able to judge effectiveness. 

Please note, also, that we need not have 97 objectives for 
reading, 74 for math, 85 for science and a few hundred others for 
pro -social behavior ©nd physical education. The behavioral objec- 
tives movement, I hope, is pai;t the day when it forced too molecular 
a view on people and trivialized teaching nd learning. We should 
not, however, throw out the baby with the oath water. Ten to 15 
objectives for reading, another dozen or so for math, and another 
dozen or so for the rest -f the currici'lum are reasonable goals to 
shoot for. For a single course in the junior high or senior high 
level a dozen or so objectives is all that is necessary to proceed to 
thinking about tffecti vness. But effectiveness cannot even be dis- 
cussed without outcomes beinq pr?v«*lent in you'- mind. 
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Outcomes -to not necessarily have to be test scores. As educa- 
tors. Me all have a legitimate interest in striving for dozens of 
outcomes for which no acceptable tests exist. For example, we usual- 
ly state that we want to develop cooperative behavior among our 
students. This Is an outcome we rightly value and for which no 
acceptable te«»ts exist. In Xhls case, effectiveness Is still judged 
by determining if the curriculum delivered is logically matched to 
the outcomes desired. You don't always need a test, you can collect 
evidence by observation. Effectiveness can be judged by observing 
classroom processes and analyzing if such processes are related to 
the Intentions of the district. A classroom that has downgraded 
competition and has Implemented cooperative learning struttures, uses 
criterion -referenced rather than norm -referenced tests, rewards 
helping behavior, etc., is one that is delivering s curriculum 
related to desired outcomes. If, however, tests are used as out- 
comes, then a concern for effectiveness means you should match the 
test to the curriculum. • If you are evaluated with a test that Is 
viewed as an Indicator of the outcomes of education, but which is hot 
matched to your curriculum, you are in deep trouble. You cannot 
possibly be effective as a school or a teacher If you have taught one 
thing but have allowed your students to be tested on something else. 

Are we educators so foolish thct we have allowed ourselves to be 
trapped In so untenable a position? Is It possible that we actually 
use tests that are not matched to our curriculum, and In that way, 
vastly underestimate the productivity and effectiveness of our 
schools'? I'd like to share with you some of the data to make 
you think about the link that has to occur between curriculum and 
outcome to even start to talk about effectiveness. 

The study I cite has to do with the match between lea'iing text- 
book series and some of the leading Indicators, the tests, that are 
used around the nation to judge educational productivity and teacher 
effectiveness. At the fourth grade level, an analysis was done of 
every item in the textbooks of the Houghton-Mifflin, Scott Foresman, 
and Addison -Wesley series and every item on the math test Included in 
five different standardized tests used to evaluate instruction at 
that grade level. The interestttig question Is: what is the overlap? 
Well, in the worst case, where a school district might be using the 
Addi son-Wesley series and using the SAT as an outcome measure, there 
was 47% overlap, meaning 53% o. the items on that test may never have 
been seen by the students before! {Figure 1) 

I come from Arizona. The school districts in my area have had 
these data since the day I gave it to them about three years ago. 
They are still using a curriculum series and a standardized test that 
do not match. We know education is slow to change, but it is time to 
stop punishing ourselves this way. We have to ask questions about 
the instrumentation we use to jud effectiveness. If your curricu- 
lum and your tests don't match, you are in trouble. In the best case 
with the Scott -Foresman series and with the MAT, 71% of what the 
students faced on the test were things they experienced in the text; 
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30% were .not. We are vastly underestimating our effectiveness when 
we allow ourselves to get trapp*?d this way. We should never have 
outcomes that don't match what we do in our classes. So, to deal with 
the effectiveness issue you have to deal with outcomes and you have 
to deal with the curriculum, and ask whether they are sensibly 
al Igned. 

Figure. 1. The Percent of Items Conmon to Both Texts and 

Tests in Regular Use in Elementary School Classes. 



Hougnton-MiffJin Scott-Foresman Addtson-Wesley 
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Hy first point is simply that the effective teacher has to have 
a match between the curriculum and the outcomes. You can't talk 
about effectiveness otherwise. Students must have the opportunity to 
learn what Is cm the test used to assess learning. But I said that 
it was the delivered curriculum that must be matched to outcomes. 
What do I mean by that? What does the term delivered mean? The way 
I use it, I define it as the involvement of students with materials, 
activities, ideas, concepts, and issues in which they have a marked 
degree of success. That to me signifies that a system is delivering 
a curriculum. Students are engaged and they are succeeding at some- 
thing. That's the delivery system. A curriculum Is delivered when 
students in a class show that they are engaged with and succeeding to 
learn what they are supposed to learn. 

Three concepts are important here, and I v<art to go over them: 
Allocated Time, Engaged Time and Success Rate. 1' e discussed these 
before. I've written about them. The work I did at the Far West 
Laboratory a number of years ago revolved around them, but 1 continue 
to speak about them because you need to think about them as you go 
dDout your work in trying to improve schools. 
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Let me start with the concept of allocated time. There is a 
positive relationship between the time allocated in instruction and 
achievement. Not every study shows it and when it's found it's not 
always a very strong predictor of achievement, but in general, the 
relationship between the time that's allocated to a curriculum area 
and the achievement in that area is positive and substantial. 

When I first told my father I was studying the relationship of 
time to achievement, he started aslcing me questions. My father was a 
bit curious about how I made a living studying how time and learning 
are related. He said, "You mean you find out that you spend more 
time and you learn more?" L said "yes", and he burst out laughing 
and said, "Boy, you've really got them all conned. Does the govern- 
ment support that?" I said, once again, "yes". He said, "I'm sure 
glad I don't pay taxes anymorel" But I think I finally turned him 
around when I explained to hiir. that the finding was not really that 
time and achievement are In some way related. The finding that is 
important, and one which every educator has to deal with. Is that the 
variability in the allocations of tliTte across classes Is enormous. 
Teachers mdke those time allocation decisions, and they are very 
important. Principals do not know what goes on behind the closed 
doors of classrooms. School superinteildents don't know; state boards 
of education don't know. Teachers are making a set of complex 
decisions. Sometimes they are wonderful decisions, sometimes they 
are not. 



TABLE 1 

ALLOCATED TIME DECISIONS m READINb AND MATHEMATICS Ih 
SELECTED GRADE TwO AND GRADE FIVE CLASSES 
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In Table 1 we present examples from our second and fifth grade 
reading and mathematics data. This is the daily time allocated to 
mathematics by the teacher in these elementary classrooms. What is 
of great interest is the variability. One teacher in the 5th grade 
allocates 20 minutes a day to mathematics, another allocates 73 
minutes a day. They're working with the same length of the school 
ddy and one is providing 20 minutes, the other 73 minutes. It's easy 
to tell which class is likely to do better on the state test of 
achievement. I really don't know how much time should be spent on 
math. I also don*t know what's the maximum time students can spend 
at math before going bonkers. I am sure, however, that 20 minutes a 
day is inappropriate. If you go into classrooms you ought to 
consider the time variable. 

With the grade five reading data tlie issue is again not how much 
time, on the average, is spent in reading (Table 1). The question 
is: What factors make one teacher allocate 68 minutes to reading and 
another one 137 minutes to ret i? Two hours of time in a curric- 
ulum area is very different t hour of time in a curriculum 
area. It's twice as much. W r ens in a school to allow that? 

The allocations of time '.u tne curriculum areas are determined 
by the elementary school teachers. At the junior highs and at the 
senior highs those times are fixed. The next question about alloca- 
tion, then, is: How much time is allocated withH the subject matter 
area to particular content areas? Both the junior and senior high 
teacher has to decide how much time to spend on two column addition, 
on quadratic equations, on ecology, or biology, etc. Again the issue 
is variation. For example, in the area of fractions, over 90 or so 
days one teacher spent zero time on fractions. Another teacher spent 
400 minutes teaching fractions to her class. Th? state required 
fractions at this grade, but this teacher did not spend any time on 
fractions. I asked the teacher at the end of the study, "You know, 
you didn't spend any time on fractions." The teacher said, "Really?" 
I said. "Yes." The teacher said, "I don't like fractions." 

My first response was anger but my se* ond response was, I think, 
much more appropriate. The fact is that not.d of us dc what it Is we 
don't want to do, and we confuse our personal decisions with our 
professional decisions. The teacher was making a personal decision--! 
don't like it so I'll leave it out. Teachers need to make profes- 
sional decisions. Very few teachers are reminded of this. A feed- 
back system is missing. If they drop an area of the curriculum out 
because they don't like it, nobody is there to notice. There Is not 
any feedback among teachers or by the instructional leaders to pre- 
vent this from happening. 

What has become clear from the research at Michigan State 
University is that the decisions made by teachers about what content 
to teach are based on three factors: how much they like the area; how 
much preparation the area requires for them; and how difficult they 
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perceive it to be for their students. Teachers hate to torture their 
students. So. if they think it's going to be difficult for their 
kids they might drop something out. Those are personal decisions. I 
am not sure they are proper professional decisions. Since nobody is 
giving teachers feedback, we are always in danger of getting 
variability like this. 

T«LE ? 

ALLOCATED TiMC OfCISIOHS IN DIFFERENT COKTENT AREAS 
FOR READING (GRADE 5) AND MATHEMATICS (GRADE i) 
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*<1aptfd frotr Dishai. 1977 (a) i 1977 (b), A rire complete discussion of these data 
maf be foontf in Berliner, 1979. 

The same is true of the reading area. The allocation of time 
there becomes important because of what is now a national concern, the 
area of comprehension. Allocated time in comprehension in these 
classrooms is presented in Table 2. In one classroom the average 
child spends 122 minutes doing translating and paraphrasing, 235 
minutes in inference and synthesis activities. These children are 
trying to read something and answer the question, w^at does It mean? 
They have to put it in their owrt words. Now, those of you who have 
been around in the last decade know that students couldn't decode 
anything back in the 1960s and up to the 1970s, In this decade, they 
decode fine; they just can't understand anything! That seems to be a 
national concern. Well, if that's true, one of the reasons that one 
might be concerned about time is whether, in fact, students are 
getting any instruction in comprehending. One of these classrooms 
manages to total about 400 minutes in comprehension activities in its 
reading and language arts program. As is seen on Table 2, another 
classroom gets 3,000 minutes of such Instruction. It's not 
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verjf difficult to figure out which classroom Is likely to do better 
on that one Item on the State test that says: "Read this paragraph 
and tell me which of the following statements is thi; ^est title for 
the paragraph." It*s a comprehension item, and 1t*s on every test 
ffOffi about third grade through college entrance. The allocation of 
time, the decisions abt^ut what content to teach and how much time to 
put In are very crucial decisions and they are often made without 
ffiu^h discussion with and feedback to teachers. 



Engaged Time 

Th# next concept that has to be dealt with' In talking about 
Instructional delivery Is the Issue of engaged time. The Issue of 
engaged time Is not very siartllng either. ^ father would be 
Incredulous, once again, \thm he said, "You mean If kids pay atten- 
tion they learn mor^r As I said, "Yes," he'd laugh again and 
wonder how you make a living telling people such startling news! The 
Issue has nothlng^ to do with that common sense finding. The Issue 
Is, do classrooms vary enormously In the mean engaged ttme recorded 
for the class? Tne answer, again. Is yes. In one classroom you find 
50% of the students engaged at any one time you are looking.* In 
another classroom 90X of the students are engaged at all times. 
Uhat's the difference betwe«) the two classrooms? What does it mean 
for the delivered curriculum? Very simply. If a teacher hasi a 50 
minute allocation to math, and 50% engagement, the children are 
getting 25 minutes a day. If there Is gO% engagement, the students 
get 45 minutes a day.' That's 20 minutes a day difference In time. 
That equals 100 minutes a week. Those are big blocks of Instruc- 
tional time. What is delivered In sheer amount of curriculum to 
students In two different classrooms may be very, very different. 

It Is Interesting to think about the delivery of curriculum and 
the time variables as follows. We can start by thinking of a school 
year, and then must say but It's not a "year". It's only 9 months or 
18U days. But then, It'f not really 169 days, because we have to 
subtract from that. What do we have to subtract? Hell, the teacher 
Is 'absent once In a while, the kids are absent once In a while, the 
teacher takes a mental health day once In a while, the buses break 
down, „snow occurs and you doirt make up the day, etc. What are we 
back to In terms of Instructional days In a district? 150 at most? « 
Let*$ say 150 Instructional days. I don't think that's uncommon. 
And let's say many of the early grades are allocating 30 minutes a 
^4y to mathematics at a 50% engagement rate. You then are delivering 
15 minutes a day of mathematics. Lets go over that again. If 30 
minutes a day Is what Is allocated for 150 days, that gives you 75 
hours of mathematics Instruction for the entire school "yea*." And 
If the engagement rate Is 50 percent, then what's delivered In 
mathematics is 37-1/2 hours, of time on tfsk. By dult standards, 
that's under one week's work. It Is not any wonder to me, when I 
think about standardized testing programs, that getting two Items 
right can give you 8 months advantage on those tests. If all we're 
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delivering In certain classrooms is 40 hours of delivered curriculum 
in math, about two items more is all you can expect for the whole 
year- 

Now, these kinds of data are not meant to say that all teachers 
are like this and that every classrooir suffers. But if you're 
talking about school effectiveness, you might want to find those 
classes where these kinds of data are obtained. Maybe it's 201 of 
the classes in 'your district; maybe it'% 15%. But whatever the 
number, those classes can be helped because these are remediable 
kinds of variables. If you want a more effective school, you've got 
to r^et more instruction delivered. It's not a very difficult concept 
to deal with. We worked in schools where we were able to peek in 
classrooms that we werfjn't ir ited to look in. All of our teachers 
were volunteers, but t^cause tiiany of them were In pod schools, you 
could sit in the center of the pod and watch the teacher who invited 
you, and all the other teachers, without tbem knowing, interest- 
ing question is: What's delivered in some of those classrooms that 
we weren't invited into? We had the Reeling that some large percent, 
somewhere between 5 and 15 percent of the school classrooms were not 
delivering lOU hours, total, in reading and math, in the elementary 
grade levels! Oo you want to tiave a more effective school system? 
Find those classrooms and provide feedback for those teachers. Most 
of them~3Tdn't know what they were actually delivering. They aren't 
used to thinking that way. These concepts help you to think about 
classrooms and, thank goodness, the numerical values of variables 
like how much instruction is being delivered are easy to change. 

The engaged time variable, we think, is very Important. After 
we finished this project, 1 went to the library to do some further 
research on what we had done. I found out that we had replicated the 
results of somebody who did this study in 1888. In fact, the issue 
of time and instruction is on a 20 year cycle. A man named Curry in 
1884, and then Judd again in 1918, and then some people in the 
193US, and Phil Jackson in the 1960s, and then some of us in the 
7Us and SUs are part ot the cyclical discovery of time variables. 
Everyone soys tim^ is Important, and every 20 years someone says not 
enouyh of it is being spent. People are concerned about time. It's 
not a magic variable; it's not a very complex variable. But the 
allocations of time to instructional areas and then the engaged time 
in those instructional areas are very important concepts with which 
to think about classrooms. 



Success Rate 

The next concept I want to deal with is that of success rate. 
Again, in our study, we found something we think gets rediscovered 
every .low, and then. That is the concept of success for young 
children. We studied whether the children were succeeding at what 
they were doing or whether the material was too difficult for them. 
Our thinking was that the match of curriculum to a child is an 



97 

JO 



Important and very complex teicher behavior. And of course teachers 
are not sadists. If ihey see children failing at something, they 
change the curriculum as quickly as possible. Nevertheless, in our 
5tudy we did find classrooms where 14% of the school day would be 
coded by our observers as high failure experiences. Students werp 
constantly failing. Would you like to have 'a predictor of failing 
performance on outcome measures? Failing performance in a classroom 
is a very good one! 

If a child can't do two column addition in a classroom, a child 
is not going to do two column addition on an outcome measure. If a 
child Succeeds at two column addition in the classroom. It's very 
likely the child will do well on a two column addition problem on the 
outcome. So, looking at the success and failure rates within a 
classroom becomes an important concern. In our study, we found that 
high success was incredibly important for young children. Jere 
Brophy and Barak Rosenshinc, researchers in this area, say that 
unless recitation activites are at the 80% success rate they're 
likely not to produce much achievement, and unless seatwork or home- 
work is up around 95X success, they may not have much value. And 
that seems particularly true in those areas of the curriculum that 
are hierarchically formed, like mathematics. If you don't really 
learn addition, you're going to have trouble with multiplication, and 
if you don't really learr. subtraction, you're going to have trouble 
with long division. There are lots of areas of the curriculum that 
are hierarchically formed, and it means that success early in the 
curriculum does determine success later in the curriculum. Our data 
are clear on this for young children. Thus success rate becomes an 
important variable for thinking about classroom instruction. 



Academic Learning Time 

I've talked about allocation of time and I've talked about 
engaged time and I've talked about success rate and I want to bring 
those together for you. in the single concept that i think Is one of 
the important ones to think about when you go into classrooms. It's 
the concept we called academic learning time, ALT. One of the pro- 
ducts of the many recent studies of teaching that I think is 
important is this ALT variable. We'va defined it in our work as 
engaged time with materials or activities that produce a high success 
rate and that are related to the outcome measures that are being 
ij&ed. It characterizes a lot about what I've said about the align- 
ment of curriculum with outcomes and about the time variables. Let 
me try to define it better with a diagram (Figure 2). Ue said that 
the allocation of time at the junior and senior high schools is 
fixed, but at the elementary schools the teachers make the decision 
of how much time to give to music, reading, math, etc. We started 
with the notion that there is an allocation of time. The next con- 
cept we analyzed was engaged time which is some subset of the 
allocated time. You don't expect these variable to be equal. That 



Figure 2, Defining Avadcmic Lwrnmy Time (ALT) 




LSR e Low Succe%» Rate ALT ^ Academic U»mlng Time 

MSft = Nfedium Succp^^ Rate 
HSR « High Su^^^ft Rate 

Interpret ^tum: The time allocated for instruction \s shown visually in (a). 
During some of thi^^ time, student^i are enga\;od, as shown in (b). Some of 
the time students are engaged is time related to the outcome measi res 
that are used to asie!>s irtstrurtion. Thi^* is shv'i.vn vi>uaKv in (c). The time 
allocjfed, wht^ther engaged or not. and ^vhelher related to the outct»mr» 
measures or not, can by yielding low, medium, ur high success* rates for 
studenS (d) That pi>rticin of aflocJted time th^t is time engaged in 
a^.tivifR'N related to the outCwWe measures .in J Ahtih provides students 
»^ ith high success rate ii» defined as Academic Le*2rmng Time, as ^hown 
in (e) 
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is. you don't expect 10U% engaged time. It's a subset. 90X would be 
nice, 85% would be nice. 201 would be bad. 

We also said that you could conceive of the success rates of 
students when you're wornlng with them; some of the time they 
are engaged they will be havlr.g high succesis; some of the time they 
are engaged they will be having medium success; and some of the time 
they will experience very low success. And you sort of hope that the 
amount of time they spend In low success activities would be small 
because failure Is not good on a regular basis. Moreover, in oor- < 
work, we found out that high success was an extremely Important ^ 
predictor of achievement. 

So these are the ccmcepts we have worked on so far, allocated 
time, engaged time, and success rate. I started off by saying 
one of the key variables was whether the curriculum is related to the 
outcome. Well we don't expect everything a teacher does to be 
related to outcomes. If we did we'd have automatons as teachers. 
There'd be no spontaneity; there'd be no fun In teaching. The 
biggest criticism of Becker and Englemann's programs, the Oistar 
programs. Is not that they dont work, because they do. The biggest 
criticism Is that they force teach'ers Into reading from manuals, 
pointing when they're supposed to, having students chant when they're 
supposed to, etc. Every part of the teacher's activity Is, In fact, 
related to the outcomes as defined by Becker and Englemann, and it 
drives a lot of teachers to drink at an early age. Not everything 
that's going to happen in classrooms Is expected to be related to the 
outcomes. If a teacher Is teaching a geometry lesson and somebody 
says, "that's a beautiful shape," it's perfectly sensible that the 
teacher would talk about how the concepts of beauty and the concepts 
of geometry were once wedded and that the history of Western civili- 
zation is based on philosophy and mathematics being Intertwined. 
Nevertheless, some part of the teaching act ought to be related to 
the outcomes. I've never seen a geometry test that asked about 
beauty. I have seen many geometry tests that check if students know 
the number of angles that are equal in an isosceles triangle. And 
the teacher is there to teach some things like that. 

Now, let's put this all together as we have in Figure 2. We say 
that some part of the allocation of time, whether it's fixed or 
variable, depending on grade level. Is engaged time. Some part of 
that time is also related to the outcomes, and some part of that time 
is high success. We're calling that Intersecting zone f^cademic 
learning time. You can measure it in minutes. You can walk into a 
classroom and get a feel for whether the class is involved m the 
things they're supposed to be. We can measure it; we call it ALT. 
ir's not easy to measure, but it's a very good predictor of achieve- 
ment. Why shouldn't It be? It says that kids are engaged. It says 
that they're succeeding at tasks like the ones that are part of the 
outcome, and that what they're doing is in fact related to the out- 
comes. It really is a very interesting vjriable. 
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How, I have to ask you to think for a moment about the variable 
of ALT again. When Me created It we didnt quite have the full 
picture in mind. We were ac':ysed^b> many educational researchers and 
psychologists of measuring the m«^^upabl«. You know, we go out with 
stop watches and we count things, and we go for the easily countable, 
the easily measurable, and we never deal with "quality." I hear that 
all the time. The ALT variable may be revolutionary, in a wav, 
because it may be the first measure we have of quality. I'd "like to 
challenge you to give me a better definition of quality instruction 
than mine. You walk into a classroom and the kids were working on 
activities or with materials or dealing with things that are related 
tc outcomes that are valued. They are engaged and they are suc- 
ceeding at what they are doing. What else do you want? It strikes 
me that if I walked Into a classroom and I could say that, I would be 
very happy. I can't hold the teacher responsible for the achieve- 
ment that's going to show up on the test, but I want to be able to 
hold my teachers responsible for the classroor. behavior I observed. 
Students should be doing things that are related to outcomes, they 
Should be succeeding at them, and they should be engaged in the 
tasks. It strikes me that that's a pretty good working definition of 
quality In Instruction when you are looking behind classroom doors. 
And I think, therefore, that the variable Is not just easily 
measurable and just dealing with time. I think It becomes a very 
Important concept for looking into classrooms. 

One of the criticisms teachers have always had of the people who 
look into their classrooms Is that they didn't know what to look for. 
That's why teachers were rated on such things as the neatness of 
their bulletin board and their clothing. I think with the advent of 
ALT, we are at the point where we can say, we now know what to look 
for. We do know what events are likely to produce achievement In 
classrooms. Is there evidence for this? Well, yes. It's not of the 
type that might overwhelm the Nobel committee, but It's of the type 
that might be very useful to an educator. 

Let me Interpret some of the data in Table X Envision three 
children who start out with a raw score on a test of 36. They're at 
the 50th percentile on our sample. These are three average kids in 
the sample we used. They then proceed to work for the next five 
weeks in their reading and we count the number of minutes each day 
that we would call ALT, Academic Learning Time. ALT is t^c^ number of 
minutes in which they are engaged in, succeeding, and working on 
tasks related to the reading outcomes. In one class, the children 
accumulated 100 minutes of ALT, in another class 573, and in another 
13O0. The average daily time, in minutes, turns out to be 4 minutes 
a day, 23 minutes a day, and 52 minutes a day, respectively. The use 
of the regression equations we created for estimation gives us the 
following Information. Five weeks later we can expect the child who 
got four minutes a day of ALT to pick up one more item on our test 
and get a score of 37. The second child gets seven more items on the 
test right, and gets a score of 43. The third child gets 16 items 
more and gets a score of 52, They've all been working at reading. 
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but their growth is different. More important, their placement in 
that sample is vastly different. Here are three children whj start 
Out average and end up, based on ALT, at the 39th percentile, the 
50th percentile, and the 66th percentile on the same kinds of tests, 
It's a pretty powerful variable. At least powerful enough to take 
seriously. 



TABLE 4 

ACADEMIC LEARNING TINE AMD STUDENT ACHIEVEMENT: 
EXAMPLES FROM GRADE 2 READING BASED ON 
BTES (PHASE III-B) RESULTS 
(A-8 PERIOD) 



Reading Score 
it 4 (October) 


Student Engaged Time 
In Reading 
with High Success Rate 


£st1nuted Reading Score 
at B (December) 


Score 

{out of 100) 


Percentile 


Total Time 
Over 5 Weeks 
(Minutes) 


Average 

Daily Time 
{Hinutes) 


Raw 

Score 

(out of 100} 


Percentile 




50 


' 100 


4 


37 


39 




SO 


S73 


23 


43 


50 




5U 


1300 


52 


52 


66 


it 


17 


lOU 


4 


20 


15 

( 


It 


17 


57i 


23 


25 


"M.. 

21 


io 


17 


130U ' 


5^ 


35 


36 



hctfS 



1 »r avprage of ^5 Stfiool days occurred between the A and B testing. 

J Tfif »a1ups of all variaMes In this table are within the ranges actually obtained In the BTES sample. 
4 Inp d»eragt' engaged time with high success rate 1r. grade 2 reading for the A-B period was 573 minutes. 



In the data I presented to you, U may appear that the range of 
4 to 52 minutes a day is unreal istically large. These values, 
however, actually occurred in the classes in our study. Furthermore, 
jOne can easily Imagine how either 4 or 52 minutes per day of acadefiiic 
learning time might come about. If 50 minutes of reading instruction 
per day is allocated to a student who pays attention only about a 
third of the time and only one fourth of the student's reading time 
Is at a high level of success, then that student will experience only 
about four minutes a day of engaged reading at a high success level. 
Similarly if 100 minutes ^er day is allocated for reading for a 
student who pays attention 85X of the time at a high level of success 
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for almost two thirds of the time, then that student will accumulate 
over '50 minutes a day of ALT, These differences In ALT do result In 
differences in achievement. 

So, I believe AuT Is a good predictor. Thus, If you need to go 
into classrooms and are worried about whether a teacher will be 
effective or not, why don't you ask some questions like the 
following: Ask If the time allocated Is adequate for the curriculum 
area or to the content areas within that curriculum area. Ask If the 
engaged time Is sufficient to ensure that what Is (Tellvered might 
actually result In learning. Ask If students appear to be successful 
In their classroom work. Ask if the work Is logically related to the 
outcomes of the course or the curriculum area. Ask If the pace Is 
sufficient to cover all the objectives held for the course or for 
that grade level because if the pace is not sufficient, students would 
not have had the opportunity to learn all they are supposed to learn. 
In Tucson, I went to a group of teachers in the same grade level who 
were working with the same Basal series and I said, "Where are you in 
the teacher's manual that goes with this series?" They said "What do 
you mean?" And, I said, "What page are you on? How much of it did 
you cover?" Well, some »ere done and had been done for a month, and 
were on to all sorts of redding enrichment programs. Some were on 
page 99 of a 40C page teacher's manual. That's pace. The raw amount 
covered would seem to be a good predictor of achievement. Why not? 
If people aren't exposed to certain things, they're not likely to do 
well on tests of those things. In the cross national comparisons 
between the United States and other countries, the single most 
important variable explaining differences in the achievement of dif- 
ferent countries was whether or not students in those countries had 
been exposed to the content. 

Then, after you ask that set of questions, you have to arm 
yourself with some observational Instruments that reflect a set of 
teaching variables that you believe in to see how those variables 
might affect the delivery of the curriculum. What teaching variables 
strould yoii Took for? What are the ones to liave faitb In? I think 
everyone has to find that out for themselves and read the literature 
for tnemselves. I'll tell you what my favorites are from my research 
and my reading of the literature. I think most of this list wvuld be 
agreed to by other people, but certainly not all of it. Structuring 
feedback, monitoring, convivial ity, handling transitions, management 
of deviance, safety, order and academic focus are some of the things 
I look for when I go intc a classroom. 

I look at structuring because it tells children what to expect. 
By st'-ucturing we mean the teacher tells the kids what they should be 
doing. Instead of the teacher saying, "O.K. spelling", the teacher 
says, "We're doing spelling. We're on page 47 of the workbook. 
Everybody get out your spelling books and those of you that are 
excused from it, here's what you're to do." They structure the 
activity. Why is this important? A couple of reasons. Kids often 
don't know what the command "spelling" means. In a highly mobile 
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society like ours, we're dealing with 50. 60 and 100 percent turnover 
in some of the urban areas. The teacher may present the ru>es for 
classrooms early In the year, but by mid-year most of those kids are 
gone and the teacher then Is assuming that the shorthand way of 
telling kids what's to be done Is understood. Some of those kids 
haven't got a clue! The teachers who structure tell students what's 
expected, where to be In the aater^al, and give directions. Teachers 
who structure seem to be those that In fact have higher achievement. 
Of course It can be abused. Ue saw one study done at Wisconsin where 
the Instructions for what to do far exceeded the doing of one 
workbook page. The teacher went on ZO minutes about how to do it and 
the kids, were through in three. There are extremes of structuring. 

I look at feedback because our research says that academic 
feedback keeps success high. It keeps children from being failures 
too long. The teacher says, "That's not correct, here's how to do 
It," A teacher who finds ways to provide academic feedback often has 
children whose success rate is higher, and that's a predictor of 
achievement. 

I look at monitoring because in today's schools, particularly 
the elementary schools, students have as much as 60% of the school 
day spent at independent work. Kids work at tables. Thsy work in 
workbooks. They work on those blu& sheets that elves produce at 
night In schools. Every morning there are stacks of elf -supplied 
worksheets. And if nobody is monitoring what children do, then it's 
not unusudl to find a second grader not doing it. What does that 
mean for the teacher? It means you have to wander the class or have 
an aide who wanders the class and keeps the monitoring rates high. 

I look at conviviality, which Is our code name for a whole bunch 
of things. Conviviality shows me that the classroom is a nice place 
to have kids. It means the attendance Is likely to be h<«jh, the 
sickness rate low. the throw -up rate low, the anxiety down. And 
that's what you want for kids. They can't work unless they feel 
comfortable and safe, 

I look at Whether a teacher handles transitions adequately. In 
our study of management of time in classrooms we had one teacher who 
was very concerned with having a modern classroom with resource 
center* all around the classroom— a listening center for reading, a 
math facts center, a social science center, the hamster center for 
studying this and that. Every 20 or 30 minutes children would be 
moving around the classroom and the teacher had what was clearly an 
exciting place to be. In a 306 minute school day, 76 minutes were 
clocked as transitional time. The kids were getting into activities 
and getting out of activities and not spending the time in the 
activity. One should look at that. The greatest transltTon problems 
are with art teachers and physical ed teachers. It takes so long to 
get ready to do art and physical ed and it takes so long to break it 
down and put it away that sometimes we actually lose art and physical 
ed. But the same is true of reading in some classes, and the same Is 
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true of mathematics. The transitions need to be handled. When I go 
out into classrooms I look at the transitions and their effect on 
engaged time, because if the transitions are too long, engaged time 
pluimets. 

I look at management of deviance and I look for the kind of 
teacher *fho has what Kovnin called with-it-ness. I look for teachers 
who can control tne classroom, who make very few timing and targeting 
errors in the way that Kovnin talks about it. Walter Borg talks about 
it, the way Emmer and Evertson talk about these concepts now in their 
handbooks. I look at whether the classroom and the school are safe, 
orderly, and academically focused because we now know you can't 
expect achievement in a place that's not orderly or that's not safe. 
An academic focus is important, but that doesn't mean the teacher 
can't laugh and joke and have a good time, and that kids can't be 
happy. It means that after the laughter and the joking, the teacher 
-knows to bring them back to the task at hand. 



biummary 

Teacher decisions and behavior affect what parts of the curricu- 
lum the student learns. They pick the tasks and the time on tasks. 
Teacher's decisions and behavior affect the success of students in 
class. Teacher's decisions and behavior affect the students' engage- 
ment and the time on the right task in classes. Teacher decisions 
and behavior affect the students' attitudes toward schools which 
affect attendance, climate, and management. In my view the teacher 
decisions and behavior only affect achievement outcomes if they work 
through student involvement and success with the right tasks. What 
are the right tasks? At least some of them are the tasks that are 
used to judge effectiveness. Thus, my original, deceptively simple 
point is stated again: Teachers who deliver a curriculum to students 
that is matched to outcomes are likely to be effective. What is nice 
IS that research has helped us find a good classroom indicator of 
whether effectiveness is likely, that is the ALT variable, con- 
sisting of allocated time, engaged time, success rate, and the 
reldtionship of activity to outcome. I hope these ideas are helpful 
ds you go out and observe classrooms. 



Research and Teacher Effectiveness 

These are good times, really. Research on teaching has given us 
A set of variables to look at that re'tate to engagement and success. 
These should be looked at closely. Research has given us an 
advantage over preceeding generations, and I think you should use 
that res^'arch wisely. The "Nay Sayers" say that our findings are 
weak, that the correlations are low, that the experimental findings 
(]on't always hold up. That may be true but that doesn't mean that 
research can't produce any useful ideas. The original study of the 
effect of cigarette smoking on cancerous conditions showed a correla- 
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tlon t>etM9en smoking and cancerous cells In the lungs of about .14. 
It's trivial. In education we say "I -^on't pay any attention to 
correlations like that." Nevertheless, It makes sense that cigarette 
smoking Is likely to produce some harm. Some of our variables in 
education make sense, even If the correlations are not high. 

I think we're short changing research. Two and three percent 
increments In our knowledge about what happens to kids are powerful 
findings. The research that we have now does that. Its strength 
is also that It is quite useful. For example, the ALT concept 
may be a way to measure something we think of as "quality." So I 
think a revolution Is taking place. Research Is on its way to help 
us In understanding the elusive nature of quality. We will In time 
be able to provide Insights Into Instruction that is both "good" and 
that Is "effective." I can't give It all to you In a brief visit 
like this, but If you'll read widely in this literature and interpret 
the literature humanly, you'll probably end up thinking of your 
teachers as handling one of the most complex environment* we've ever 
asked anybody to face. 

How do you take care of pace and make sure that everybody gets 
through, while at the same time you're keeping success rate high? 
How do you keep engagement high when you've been given malnstreamed 
kids and you have to pay special attention tc one or two kids in your 
classroom? We've asked the impossible of teachers. The research 
findings are going to have to be filtered through you, as instruc- 
tional leaders, and be used humanly in studying the context that 
you're walking Into. The Issue of pace may have to be abandoned for 
part of the school term as the teacher is just simply establishing 
order. The issue of success rate may have to go by the boards as the 
teachers bring something else into li'^e. Many teachers say, "I wish 
you'd come visit in November, I don't get my class started until 
then." They've had them since September, but they may be right. It 
takes time to get these systems going, and we can't just apply 
research evidence and say why aren't you teaching fractions? It may 
be Inappropriate at that time. You have to check with your teachers 
and use them as guides, too. I've 1ea'*ned to do that as I've talked 
to them. If you think about the humane use of the research, I think 
yoo will find that the research can help you Improve your schools. 
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